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ABSTRACT 


Prognostic  and  Health  Management  (PHM)  systems  often  experience  delayed  fielding 
and  lengthened  maturation  cycles  due  to  their  relative  immaturity  and  the  fact  that  they 
are  regarded  as  non- flight  critical  systems.  The  national  fiscal  crisis  and  rising  debt  of  the 
U.S.  have  each  placed  increased  scrutiny  on  military  systems  acquisition  and 
procurement  practices.  The  Defense  Department  is  pushing  for  greater  emphasis  on 
fundamental  systems  engineering  practices  earlier  in  the  acquisition  phase,  with  the 
expectation  of  fewer  schedule  slips  and  budget  overruns.  The  acquisition  of  PHM 
systems  could  also  benefit  from  increased  systems  engineering  rigor  early  in  their 
development.  A  2007  directive  from  the  DoD  states  that  PHM  systems  be  implemented 
into  current  weapon  systems  equipment,  and  materiel  sustainment  programs  where 
technically  feasible  and  beneficial.  This  research  examines  the  definition  of  PHM 
requirements  and  a  method  for  developing  a  solution  neutral  architecture  for  PHM 
systems.  The  thesis  also  identifies  software  development  practices  and  acquisition 
processes  for  military  propulsion  PHM  systems.  The  conclusion  of  this  research  is  that 
the  Defense  Department  can  deliver  the  warfighter  a  capable  PHM  system  on-time  and 
within  budget  through  the  establishment  of  better  procurement  and  systems  engineering 
practices. 
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EXECUTIVE  SUMMARY 


PHM  systems  are  meant  to  replace  legacy  schedule-based  maintenance  systems  and  it  is 
important  for  acquisition  professionals  to  manage  customer  expectations.  The  research 
will  show  that  it  is  the  responsibility  of  program  managers  and  systems  engineers  to  field 
capable  CBM+  systems  early  in  the  weapon  system  life  cycle,  such  that  the  system  and 
the  overarching  CONOPS  can  be  optimized  and  matured.  Some  keys  to  delivering 
capable  PHM  systems  closer  to  budget  and  cost  targets  are: 

1 .  Incentivize  Program  Management  to  focus  on  Life  Cycle  Cost  Savings 

2.  Involve  the  end-user  beyond  the  requirements  development  phase 

3.  Manage  warfighter  expectations  to  minimize  Taguchi  losses 

4.  Plan  to  support  PHM  software  throughout  the  weapon  system  life  cycle 

This  research  provides  a  useful  reference  to  program  managers  and  systems 

engineers  tasked  with  the  implementation  of  Prognostic  and  Health  Management  (PHM) 
systems.  It  presents  the  acquisition  of  these  systems  within  the  framework  of  systems 
engineering  process  and  also  provides  acquisition  strategies  for  PHM  hardware  and 
software  products. 

The  establishment  of  stakeholder  requirements  for  PHM  systems  is  enabled  by  an 
understanding  of  legacy  maintenance  practices  and  an  assessment  of  the  technologies 
currently  available  to  replace  schedule-based  maintenance  practices.  This  research 
examines  these  maintenance  practices  and  presents  a  strategy  for  evaluating  PHM 
technologies. 

The  next  step  in  this  systems  engineering  analysis  of  a  PHM  system  is  the 
establishment  of  a  notional  architecture.  This  solution  neutral  approach  helps  us  model 
the  system  without  any  pre-conceived  notions  of  how  to  solve  the  problem.  Once  this 
high-level  architecture  is  presented  the  research  discusses  software  development.  Systems 
engineering  processes  and  an  evolutionary  approach  to  software  development  are 
identified  as  necessary  elements  to  an  on-time  and  on-budget  delivery. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Prior  to  the  F-35  Joint  Strike  Fighter  (JSF)  Program,  the  A-7  Corsair  II  was  the 
last  single  engine  fighter  in  the  Navy  tactical  aviation  fleet.  According  to 
GlobalSecurity.org,  “the  A-7  began  combat  operations  in  December,  1967,  and  proved  to 
be  one  of  the  most  effective  Navy  close  support  and  strike  aircraft  during  the  Vietnam 
War  [1].”  The  A-7  was  also  the  platform  that  marked  the  beginning  of  on-board  engine 
health  monitoring  for  the  Navy.  A  brief  given  by  Andrew  Hess  at  an  NDIA  Conference  in 
2001  states,  “the  engine  monitoring  system  (EMS),  developed  by  Navy  engineers, 
tracked  engine  parameters  and  used  two  vibration  transducers  to  detect  anomalies  in  the 
mechanical  health  of  the  engine.  The  EMS  was  a  successful  technological  advancement 
on  the  A-7  and  eventually  reduced  engine  related  accident  rates  by  90%  and  reduced 
maintenance  man  hours  [2].” 

The  Health  and  Usage  Monitoring  System  (HUMS)  developed  for  military 
helicopters  greatly  advanced  the  state  of  mechanical  diagnostics.  This  system  architecture 
was  mainly  composed  of  vibration  sensors  mounted  on  the  propulsion  system  and 
throughout  rotary  wing  drivetrains.  The  current  engine  monitoring  system  being 
developed  for  the  JSF  is  known  as  Prognostics  and  Health  Management  (PHM). 
According  to  their  website,  “the  JSF  Program  has  leveraged  advances  in  sensing 
technologies  and  processing  capabilities  to  develop  an  integrated  engine  monitoring 
system  with  the  ability  to  detect  impending  failures  well  in  advance,  thus  increasing 
fleetwide  safety  and  decreasing  operating  costs  [3].” 

The  Navy  has  had  an  increased  focus  and  commitment  to  advancing  the 
technological  capability  of  engine  monitoring  systems  in  each  air  system  procured  since 
the  A-7.  Figure  1  loosely  illustrates  the  evolution  of  diagnostic  systems  in  Navy  aircraft 
since  the  late  1960s. 
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Figure  1.  Evolution  of  EMS  in  Naval  Aviation.  From  [2] 

A  DoD  Instruction  titled  Condition  Based  Maintenance  Plus  for  Materiel 
Maintenance  states  that  “CBM+  is  the  application  and  integration  of  appropriate 
processes,  technologies,  and  knowledge-based  capabilities  to  improve  the  reliability  and 
maintenance  effectiveness  of  DoD  systems  and  components  [4].”  The  instruction  also 
mandates  that  all  acquisition  programs  must  implement  a  CBM+  strategy  and  concept  of 
operation  (CONOPS)  “into  current  weapon  systems,  equipment,  and  materiel  sustainment 
programs  where  technically  feasible  and  beneficial  [4].”  For  the  Navy  and  their  aviation 
propulsion  systems,  the  implementation  of  this  DoD  Instruction  will  eliminate  scheduled 
inspections  of  turbomachinery  and  enable  the  health  assessment  of  components  based  on 
their  actual  usage  and  life  remaining.  This,  in  turn,  will  lower  sustainment  costs  and 
increase  fleet  readiness. 
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B. 


PURPOSE 


The  largest  life  cycle  cost  associated  with  military  aircraft  is  not  the  unit  price  or 
the  cost  of  the  multi-year  development  programs:  rather,  it  is  operation  and  sustainment 
(O&S)  costs.  The  life  cycle  of  military  aircraft  varies  by  model,  but  it  is  usually  no  less 
than  25  years.  The  cost  associated  with  operating  and  supporting  these  aircraft  is  usually 
between  65%  to  80%  of  the  entire  life  cycle  cost. 

The  DoD  has  recognized  the  need  to  control  O&S  costs  and  is  taking  steps  to 
reduce  them  for  future  aviation  programs.  Figure  2  illustrates  that  O&S  comprises  the 
largest  portion  of  the  weapon  system  lifecycle.  It  is,  therefore,  the  area  with  the  most 
potential  for  cost  savings. 


Procurement  Funds 


o 


Operations  &  Maintenance  Funds 


Figure  2.  Weapon  System  Life  Cycle  Illustrates  that  O&S  Represents  a  Majority  of 
Life  Cycle  Costs.  From  [5] 

The  DoD  mandate  to  implement  PHM  technologies  and  a  CBM+  strategy  are 
meant  to  specifically  target  cost  savings  in  this  area.  The  benefits  of  the  system  go 
beyond  O&S  cost  savings,  however.  According  to  Andrew  Hess,  “PHM  can  also  enhance 
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mission  reliability  and  aircraft  safety,  eliminate  scheduled  inspections,  eliminate  “cannot 
duplicates  and  retest  OK’s,”  reduce  aircraft  downtime,  provide  opportunistic 
maintenance,  and  detect  incipient  faults  [6].” 


Over  the  past  decade,  there  has  been  a  substantial  increase  in  the  amount  of 
military  spending.  Figure  4  illustrates  the  trend  in  defense  spending  since  2001.  The  large 
number  of  new  defense  acquisition  programs  has  brought  with  it  a  greater  scrutiny  of  the 
cost  overruns  and  schedule  delays  that  have  plagued  weapon  systems  acquisition. 


Budget  Authority  for  National  Defense,  FY  2001  -201 1 
(in  billions  of  constant  FY10  dollars) 


War  funding 
(OCO  and  War 
Supplemental ) 

|  l>ot.  and  Other 
Defense  Related 
Funding  (053 
054) 

|  Dol)  Base  (051) 


FY0I  FY02  FY03  FY04  FY05  FY06  FY07  FY0X  FY09  FY10  FY1 1 

cst.  cst. 

Fiscal  Year  (FY) 

Figure  3.  Defense  Spending  Profde.  From  [7] 


The  DoD  has  placed  a  greater  emphasis  on  fundamental  systems  engineering 
practices  earlier  in  the  acquisition  phase,  with  the  expectation  of  fewer  schedule  slips  and 
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budget  overruns.  Legacy  acquisition  programs  have  shown  that  PHM  systems  often 
experience  delayed  fielding  and  lengthened  maturation  cycles.  The  current  acquisition 
strategies  used  by  the  DoD  could  benefit  from  increased  systems  engineering  scrutiny 
and  rigor  early  in  their  development.  The  purpose  of  this  research  is  to  investigate  how 
PHM  systems  and  architectures  have  evolved  and  identify  ways  to  leverage  systems 
engineering,  program  management,  and  systems  acquisition  practices  to  procure  and  field 
more  effective  PHM  system. 

This  thesis  is  meant  to  provide  Systems  Engineers  and  Program  Managers  a 
resource  for  the  acquisition  of  more  effective  propulsion  PHM  systems.  The  research 
outlines  processes  for  establishing  requirements,  defining  a  notional  architecture  and 
developing  software  for  PHM  systems  and  presents  them  within  the  systems  engineering 
process.  It  also  provides  acquisition  strategies  and  lessons  learned  based  on  legacy  PHM 
systems.  These  systems  are  required  for  many  DoD  weapons  and  improved  systems 
engineering  processes  tailored  for  PHM  systems  acquisition  will  improve  cost  and 
schedule. 

C.  RESEARCH  QUESTIONS 

This  thesis  will  examine  the  systems  architecture  evolution  of  military  aircraft 
propulsion  diagnostic  systems.  The  analysis  will  start  with  early  military  aircraft 
diagnostic  systems  and  continue  through  the  current  Prognostic  and  Health  Management 
(PHM)  systems  currently  being  developed  and  fielded.  The  thesis  will  also  identify 
heuristics  and  strategies  to  improve  acquisition  strategies  for  procuring  PHM  systems. 
The  following  research  questions  will  provide  a  roadmap  for  answering  the  overarching 
research  question. 

1 .  How  do  we  establish  requirements  for  propulsion  PHM  systems? 

2.  What  process  can  we  use  to  define  a  high-level  solution  neutral  PHM 
system  architecture? 

3.  What  systems  engineering  process  can  be  used  to  develop  software  for 
PHM  systems? 
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4.  Can  the  Defense  Department  deliver  the  Warfighter  a  capable  PHM 
system  on-time  and  within  budget  through  the  establishment  of  better 
procurement  and  systems  engineering  practices? 

D.  BENEFITS  OF  STUDY 

This  research  will  be  used  to  recommend  improvements  to  the  acquisition  of 
PHM  systems.  The  research  will  identify  fundamental  systems  engineering  aspects  in  the 
procurement  of  PHM  systems  and  offer  ways  to  field  more  capable  systems  earlier  in  the 
weapon  system  lifecycle. 

E.  SCOPE 

The  focus  of  this  thesis  is  limited  to  propulsion  PHM  systems  for  fixed  wing 
tactical  fighters.  This  research  will  employ  systems  engineering  processes  and  tools  to 
identify  ways  to  maximize  the  value  and  capability  of  these  PHM  systems. 

F.  METHODOLOGY 

The  needs  analysis  phase  developed  by  Kossiakoff  defines  the  methodology  used 
in  approaching  the  research.  Kossiakoff  describes  the  needs  analysis  as  “part  of  the 
conceptual  exploration  phase;”  it  was  therefore  useful  in  developing  an  outline  for  this 
research  [8].  The  operational  deficiencies  and  technical  opportunities,  as  illustrated  in 
Figure  5,  are  inputs  to  the  needs  analysis. 
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Figure  4.  Kossiakoff  -  Needs  Analysis.  From  [8] 

For  the  purposes  of  this  research,  the  operational  deficiencies  represent  the  legacy 
maintenance  practices  used  to  maintain  DoD  propulsion  systems  and  the  technical 
opportunities  are  the  CBM+  systems  meant  to  replace  them.  The  legacy  maintenance 
practices  are  only  operationally  deficient  in  the  sense  that  there  exists  a  mandate  to 
replace  them  with  technology  driven  solutions  that  whereby  the  DoD  can  potentially 
realize  O&S  cost  savings.  This  research  will  identify  methods  for  leveraging  these 
technological  opportunities  using  systems  engineering  best  practices.  In  Chapter  II,  a 
requirements  analysis  of  PHM  systems  is  developed.  This  section  uses  System  Studies  to 
examine  legacy  systems  that  have  implemented  PHM  systems.  The  research  will  also 
perform  a  Technology  Assessment  of  the  PHM  sensors  and  examine  how  the  F-35 
program  used  this  assessment  to  downselect  its  sensing  suite.  The  synthesis  of  these 
concepts  provides  a  foundation  to  establish  some  notional  requirements  for  a  PHM 
system.  In  Chapter  III,  the  architecture  for  this  notional  system  is  developed  along  with 
software  development.  This  system  architecture  represents  the  output  of  the  needs 
analysis  phase,  which  then  leads  to  Concept  Exploration.  The  concept  in  this  block  is 
represented  by  the  final  research  question  presented  in  the  previous  section:  Can  the 
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Defense  Department  deliver  the  Warfighter  a  capable  PHM  system  on-time  and  within 
budget  through  the  establishment  of  better  procurement  and  systems  engineering 
practices?  Chapter  IV  will  examine  these  process  and  practices  and  identify  some  of  the 
issues  associated  with  the  acquisition  of  PHM  systems.  Chapter  V  will  compile  the 
research  and  provide  heuristics  for  future  acquisition  programs. 

This  research  also  follows  a  framework  similar  to  the  PD-21  curriculum  at  the 
Naval  Postgraduate  School.  The  program  initially  focused  on  systems  engineering 
principles  and  concepts,  while  the  latter  part  was  dedicated  to  the  author’s  chosen 
concentration  in  systems  acquisition.  The  first  three  chapters  of  this  research  provide  a 
systems  engineering  foundation  for  PHM  systems  and  the  overarching  CBM+ 
architecture  they  are  meant  to  support.  The  final  two  chapters  integrate  these  systems 
engineering  concepts,  but  focus  on  coursework  related  to  the  acquisition  concentration. 
Overall,  the  research  is  meant  to  explore  the  design,  development,  and  acquisition  of 
PHM  systems,  while  applying  the  concepts  and  principles  presented  during  the  PD-21 
coursework. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

The  research  in  this  thesis  builds  upon  concepts  presented  in  various  sources. 
However,  the  DoD  CBM+  guidebook  provides  the  foundation  for  the  requirements  and 
architecture  sections  of  this  research.  The  foreword  of  the  guidebook,  provided  by  Jack 
Bell,  Deputy  Undersecretary  of  Defense  for  Logistics  and  Materiel  Readiness,  states  that 
“it  was  developed  to  be  an  information  reference  as  well  as  a  tool  to  assist  logistics 
managers  with  CBM+  project  development,  implementation,  and  execution  [5].”  This 
thesis  is  also  meant  to  be  an  information  reference  and  tool  to  assist  the  developers  of 
PHM  systems.  This  research  will  leverage  the  processes  in  the  CBM+  Guidebook, 
present  them  within  the  systems  engineering  process  and  tailor  them  to  PHM  Systems. 

According  to  DoD  Instruction  4151.22,  “CBM+  is  maintenance  performed  based 
on  evidence  of  need  provided  by  Reliability  Centered  Maintenance  (RCM)  analysis  and 
other  enabling  processes  and  technologies.  CBM+  uses  a  systems  engineering  approach 
to  collect  data,  enable  analysis,  and  support  the  decision-making  processes  for  system 
acquisition,  sustainment,  and  operations  [4].”  The  systems  engineering  approach  used  by 
CBM+  is  also  discussed  with  greater  detail  in  the  CBM+  Guidebook.  It  states  that 
“systems  engineering  is  the  overarching  process  that  a  program  team  applies  to  move 
from  a  required  capability  to  an  operationally  effective  and  suitable  system.  Systems 
engineering  processes  are  applied  early  in  concept  refinement,  and  then  continuously 
applied  throughout  the  system’s  life  cycle.  Program  managers  and  life-cycle  logisticians 
should  consider  the  effect  system  development  decisions  (such  as  the  application  of  the 
CBM+  strategy)  will  have  on  the  long-term  operational  effectiveness  and  the  logistics 
affordability  of  the  system.  The  cost  to  implement  a  system  change,  including 
supportability  enhancements,  increases  as  a  program  moves  further  along  its  life  cycle. 
CBM+  has  the  greatest  leverage  in  the  early  stages  of  development,  when  the  program 
design  is  most  flexible  [5].”  This  research  aims  to  explore  the  early  stages  of 

development  for  PHM  systems  where  flexibility  provides  system  developers  the 
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opportunity  to  perform  trades  on  various  design  criteria.  For  the  purposes  of  this  research, 
these  early  stages  include  the  requirements  and  architecture  definition. 

The  CBM+  guidebook  also  states  that  it  “presents  key  elements  and 
implementation  strategies  for  achieving  incorporation  of  CBM+  enablers  into  the  DoD 
maintenance  process  [5].”  It  is  important  to  understand  how  PHM  fits  within  the  system 
of  systems  that  comprise  CBM+.  The  following  sections  discuss  concepts  presented  in 
the  DoD  CBM+  Guidebook  as  well  as  several  other  related  sources  to  provide  a  baseline 
understanding  of  the  maintenance  concepts  upon  which  this  research  builds. 

B.  CONDITION  BASED  MAINTENANCE  PLUS  (CBM+) 

The  CBM+  guidebook  states  that  “CBM+  is  not  a  single  process  in  itself.  It  is  a 
comprehensive  strategy  to  select,  integrate,  and  focus  a  number  of  process  improvement 
capabilities,  thereby  enabling  maintenance  managers  and  their  customers  to  attain  the 
desired  levels  of  system  and  equipment  readiness  in  the  most  cost  effective  manner 
across  the  total  life  cycle  of  the  weapon  system.  CBM+  includes  a  variety  of  interrelated 
and  independent  capabilities  and  initiatives — some  procedural  and  some  technical  that 
can  enhance  basic  maintenance  tasks  [5].” 

A  discussion  of  components  of  CBM+,  as  found  in  Figure  5,  provides  necessary 
background  for  the  research  presented  in  subsequent  chapters.  Reliability  centered 
maintenance  is  another  process  used  by  the  DoD  to  enable  informed  maintenance 
practices.  The  Office  of  the  Secretary  of  Defense  defines  RCM  as  “a  logical,  structured 
process  used  to  determine  the  optimal  failure  management  strategies  for  any  system, 
based  on  system  reliability  characteristics  and  the  intended  operating  context.  RCM 
defines  what  must  be  done  to  a  system  to  achieve  the  desired  levels  of  safety,  reliability, 
environmental  soundness,  and  operational  readiness,  at  best  cost.  RCM  is  to  be  applied 
continuously  throughout  the  life  cycle  of  any  system  [9].” 

In  his  paper  Defining  Requirements  for  Advanced  PHM  Technologies  for  Optimal 
Reliability  Centered  Maintenance,  Dr.  Richard  Millar  states  that  “RCM  provides  a 
logical  decision  process  for  determining  optimum  maintenance  approaches  and 
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establishes  the  evidence  of  need  for  both  reactive  and  proactive  maintenance.  RCM  is 
used  along  with  the  propulsion  system  FMECA1  to  identify  capability  gaps  in  the  fault 
detection.  Given  a  reasonably  representative  FMECA  and  a  baseline  suite  of  proven 
PHM  tools,  a  rigorous  RCM  analysis  should  yield  PHM  functional  requirements  and 
preferred  state  of  the  art  technical  approaches.  If  this  fails  to  meet  weapon  system 
objectives,  we  may  need  to  make  tradeoffs  between  safety,  availability,  initial  and 
recurring  costs,  and  weapon  system  performance  [10].”  Figure  5  illustrates  the  means  by 
which  CBM  and  RCM,  combined  with  systems  engineering  principles  and  PHM 
capabilities,  form  CBM+. 
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Figure  5.  Various  Systems  and  Processes  that  Comprise  Condition  Based 
Maintenance  Plus.  From  [5] 


1  FMECA — Failure  Mode  Effects  and  Criticality  Analysis.  Process  used  to  determine  the  functions, 
functional  failures,  and  failure  modes  of  equipment;  and  the  associated  effects,  severity,  and  frequency  of 
each  failure  modes  [11]. 
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The  CBM+  guidebook  states  that  CBM+  includes,  but  is  not  limited  to,  the  following 
examples: 

•  Hardware — system  health  monitoring  and  management  using  embedded 
sensors;  integrated  data  bus 

•  Software — decision  support  and  analysis  capabilities  both  on  and  off 
equipment;  appropriate  use  of  diagnostics  and  prognostics;  automated 
maintenance  information  generation  and  retrieval 

•  Design — open  system  architecture;  integration  of  maintenance  and 
logistics  information  systems;  interface  with  operational  systems; 
designing  systems  that  require  mini-  mum  maintenance;  enabling 
maintenance  decisions  based  on  equipment  condition 

•  Processes — RCM  analysis;  a  balance  of  corrective,  preventive,  and 
predictive  maintenance  processes;  trend-based  reliability  and  process 
improvements;  integrated  infonnation  systems  providing  logistics  system 
response;  Serialized  Item  Management 

•  Communications — databases;  off-board  interactive  communication  links 

•  Tools — integrated  electronic  technical  manuals  (i.e.,  digitized  data); 
automatic  identification  technology;  item-unique  identification;  portable 
maintenance  aids;  embedded,  data-based,  interactive  training 

•  Functionality — low  ambiguity  fault  detection,  isolation,  and  prediction; 
optimized  maintenance  requirements  and  reduced  logistics  support 
footprints;  configuration  management  and  asset  visibility  [5]. 

The  Defense  Acquisition  Guidebook  for  Systems  Engineering  states  that  CBM+ 
is  a  system  of  systems  (SoS)  that  presents  a  fundamental  systems  engineering  problem: 
“the  integration  of  various  and  disparate  technologies  and  concepts  [12].”  It  defines  an 
SoS  as  a  “set  or  arrangement  of  systems  that  results  from  independent  systems  integrated 
into  a  larger  system  that  delivers  unique  capabilities.  Both  systems  and  SoS  confonn  to 
the  accepted  definition  of  a  system,  in  that  each  consists  of  parts,  relationships,  and  a 
whole  that  is  greater  than  the  sum  of  its  parts  [12].” 

C.  ESTABLISHING  REQUIREMENTS  AND  ARCHITECTURE  FOR  PHM 

SYSTEMS 

In  2001,  the  Defense  Acquisition  University  published  a  Systems  Engineering 
Fundamentals  (SEF)  book  that  “provides  a  basic,  conceptual-level  description  of 

engineering  management  disciplines  that  relate  to  the  development  and  life  cycle 
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management  of  a  system  [13].”  The  SEF  provides  a  roadmap  to  support  the  development 
of  requirements  definition  for  PHM  systems.  The  SEF  states  that  “requirements  are  the 
primary  focus  in  the  systems  engineering  process  because  the  process’s  primary  purpose 
is  to  transform  the  requirements  into  designs  [13].”  Figure  6  represents  the  systems 
engineering  process  and  illustrates  how  requirements  are  the  basis  of  design  through 
iterative  processes. 


Figure  6.  Illustrates  the  Systems  Engineering  Process.  From  [13] 


To  establish  stakeholder  driven  requirements  for  a  PHM  system  that  will  enable  a 
CBM+  conops,  program  managers  and  systems  engineers  must  understand  the  current 
schedule  based  maintenance  practices.  The  CBM+  guidebook  states  that  “maintenance 
can  be  performed  using  a  wide  variety  of  approaches.  Two  main  categories  of 
maintenance,  reactive  and  proactive,  encompass  the  full  range  of  options  available: 
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•  Reactive  maintenance  (also  called  corrective  maintenance)  is  performed 
for  items  that  are  selected  to  run  to  failure  or  those  that  fail  in  an 
unplanned  or  unscheduled  manner.  An  item  may  be  on  a  schedule  for 
periodic  maintenance,  but  if  it  fails  prematurely,  it  will  require 
maintenance  to  fix.  Reactive  maintenance  of  a  reparable  item  is  almost 
always  unscheduled  in  the  sense  the  failure  occurred  unpredictably. 
Reactive  maintenance  restores  an  item  to  a  serviceable  condition  after  the 
failure  has  occurred. 

•  Proactive  maintenance  is  considered  either  preventive  or  predictive  in 
nature,  and  the  maintenance  performed  can  range  from  an  inspection,  test, 
or  servicing  to  an  overhaul  or  complete  replacement: 

•  Preventive  or  scheduled  maintenance  can  be  based  on  calendar 
time,  equipment  operating  time,  or  a  cycle  (such  as  number  of 
starts,  air  vehicle  landings,  rounds  fired,  or  miles  driven). 
Preventive  maintenance  may  be  either  scheduled  or  unscheduled; 
that  is,  it  is  initiated  based  on  predetennined  intervals  or, 
alternatively,  triggered  after  detection  of  a  condition  that  may  lead 
to  failure  or  degradation  of  functionality  of  the  weapon, 
equipment,  or  component. 

•  Predictive  maintenance  can  be  categorized  as  either  diagnostic  or 
prognostic.  Diagnostic  identifies  an  impending  failure,  while 
prognostics  add  the  capability  to  forecast  the  remaining  equipment 
life.  Knowing  the  remaining  life  is  an  obvious  benefit  to  enable 
optimum  mission  and  maintenance  planning  [5].” 

Figure  7  provides  a  breakdown  of  reactive  and  proactive  maintenance  approaches. 
Based  on  this  table,  it  is  obvious  that  the  DoD  would  want  to  limit  reactive  maintenance 
on  costly  man-in-the-loop  weapon  systems.  There  are,  however,  many  variables  within 
the  decision  between  preventative  and  predictive  maintenance  approaches.  The  predictive 
systems  are  complex  and  expensive,  but,  as  discussed  above,  the  DoD  has  begun  to 
pursue  this  approach  due  to  the  opportunity  for  O&S  cost  savings.  The  use  of  reactive 
and  proactive  maintenance  approaches  and  how  they  relate  to  military  propulsion  PHM 
systems  is  presented  later  in  this  research. 
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Maintenance  need  is 
projected  as 
probable  within 
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analysis 
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real-time  trend 
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Figure  7.  Breakdown  of  Reactive  and  Proactive  Maintenance  Approaches.  From  [5] 


The  CBM+  guidebook  states  that  “the  most  difficult  task  for  the  CBM+ 
implementation  team  may  be  to  correctly  match  available  hardware,  software,  and 
supporting  technology  solutions  to  the  requirements  of  the  future  maintenance  process. 
This  task  must  begin  with  the  documentation  of  functional  requirements  [5].”  Having 
established  a  baseline  for  legacy  maintenance  practices,  the  technical  assessment  of 
available  capabilities  is  the  next  step  in  establishing  requirements  for  a  PHM  system.  The 
CBM+  guidebook  suggests  that  a  “proof  of  principle  demonstration”  is  often  the  best 
approach  to  evaluating  technologies.  The  guidebook  states  that  “in  light  of  the  time  and 
funding  resources  required  for  CBM+  implementation,  it  is  highly  advisable  for 
implementers  to  accomplish  small-scale  demonstrations  of  primary  CBM+  methods  and 
technologies  before  full-scale  implementation.  A  short-term  pilot  test  that  uses  equipment 
likely  to  be  used  for  later  full  implementation  can  be  a  low  risk  approach  to  ensuring  the 
feasibility  and  benefits  of  the  desired  capabilities.  Demonstration  of  CBM+  planned 
methods  and  technologies  give  managers  a  higher  degree  of  confidence  in  the  likelihood 
of  future  success  [5].”  In  the  following  chapter,  this  research  explores  a  PHM  acquisition 
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program  that  used  a  proof  of  principle  approach  to  evaluate  PHM  technologies  and 
discusses  how  this  test  enables  the  informed  development  of  functional  requirements. 

Once  the  requirements  are  defined  for  the  PHM  system,  the  next  step  in  the 
systems  engineering  process  illustrated  in  Figure  6  is  functional  analysis  and  allocation. 
The  SEF  states  that  “system  requirements  are  allocated  and  defined  in  sufficient  detail  to 
provide  design  and  verification  criteria  to  support  the  integrated  system  design.  This  top- 
down  process  of  translating  system  level  requirements  into  detailed  functional  and 
performance  design  criteria  include: 

•  Defining  the  system  in  functional  tenns,  and  then  decomposing  the  top- 
level  functions  into  sub-functions.  That  is,  identifying  at  successively 
lower  levels  what  actions  the  system  has  to  do, 

•  Identifying  and  defining  all  internal  and  external  functional  interfaces, 

•  Identifying  functional  groupings  to  minimize  and  control  interfaces 
(functional  partitioning), 

•  Revisiting  the  requirements  analysis  step  as  necessary  to  resolve 
functional  issues  [13].” 

A  functional  allocation  for  a  propulsion  PHM  system  is  performed  in  Chapter  II 
using  methods  outlined  in  the  SEF  to  provide  a  solution  neutral  architecture. 

The  requirements  and  architecture  development  in  the  following  chapter  will 
build  upon  these  established  concepts  presented  above.  The  literature  research  for  the 
software  development  and  acquisition  strategy  is  woven  into  Chapters  IV  and  V.  Much  of 
the  material  in  these  chapters  is  based  on  the  PD-21  course  work  and  the  Author’s  lessons 
learned  having  worked  PHM  acquisition  at  NAY AIR  since  2003. 
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III.  PHM  REQUIREMENTS  AND  ARCHITECTURE 


A.  INTRODUCTION 

This  chapter  explores  the  definition  of  requirements  for  a  propulsion  PHM  system 
within  a  systems  engineering  process.  In  order  to  frame  a  discussion  about  systems 
engineering,  it  would  be  useful  to  start  with  a  commonly  accepted  definition  from 
Professor  Gary  Langford  at  the  Naval  Postgraduate  School: 

Systems  engineering  is  a  systematic,  disciplined  approach  to  define 

stakeholder-driven  requirements  and  to  satisfy  those  with  the  development 

of  functionally  driven,  synergistic,  innovative  alternatives  [14]. 

This  definition  can  be  decomposed  to  two  basic  parts:  requirements  and 
functional  allocation.  In  order  to  define  stakeholder  driven  requirements  for  a  PHM 
system,  it  is  necessary  to  examine  the  legacy  maintenance  CONOPS.  This  activity  helps 
bridge  the  gap  between  legacy  maintenance  practices  and  those  needed  to  enable  a 
CBM+  conops.  This  chapter  examines  schedule-based  inspections  and  maintenance 
actions  currently  used  on  legacy  propulsion  systems.  An  understanding  of  these  processes 
and  procedures  provides  a  foundation  for  establishing  requirements  for  a  PHM  system. 

Once  the  legacy  practices  are  established,  this  chapter  addresses  the  assessment  of 
PHM  technologies.  One  of  the  most  important  and  fundamental  roles  of  a  Systems 
Engineer  is  to  understand  both  currently  available  and  emerging  technologies.  Situational 
awareness  of  emerging  technologies  will  enable  a  program  to  incrementally  integrate 
capabilities  throughout  the  development  of  the  weapon  system.  The  integration  of  the 
legacy  CONOPS  and  emerging  technologies  provides  a  basis  for  the  discussion  of 
requirements  and  functional  allocation  for  a  PHM  system. 

This  chapter  also  outlines  a  high-level  approach  to  establishing  systems 
architecture  for  a  PHM  system.  This  solution-neutral  approach  will  apply  principles  of 
systems  architecting  to  the  design  of  propulsion  PHM  system. 
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B.  LEGACY  MAINTENANCE  CONOPS 

As  stated  in  Chapter  II,  the  legacy  CONOPS  for  maintaining  tactical  fighter 
aircraft  propulsion  systems  is  either  reactive  or  proactive.  The  redundant  and  safety 
critical  design  of  propulsion  system  components  addresses  many  of  the  failures  that 
would  drive  reactive  maintenance.  However,  some  system  faults  that  require  reactive 
maintenance  cannot  be  designed  out  of  a  propulsion  system,  such  as  foreign  object 
damage  (FOD)2  and  maintenance  induced  faults.3  Despite  the  occurrence  of  some 
reactive  maintenance,  the  vast  majority  of  maintenance  performed  on  aircraft  propulsion 
systems  is  proactive.  These  preventive  and  predictive  inspections  generally  take  place  at 
intervals  determined  by  engine  flight  hours  or  sorties.  Some  typical  proactive 
maintenance  actions  for  tactical  aviation  propulsion  systems  include:  borescope 
inspections,  duct  dives,  and  oil  sampling  activities.  These  activities  are  labor  intensive 
and  over  the  life  cycle  of  an  aircraft,  these  inspections  can  drive  a  great  deal  of  O&S 
costs. 

Borescope  inspections  in  legacy  aircraft  are  generally  perfonned  based  on 
accumulated  flight  hours.  The  inspection  can  be  somewhat  invasive  and  time  consuming 
on  some  legacy  platforms  due  to  the  location  of  the  borescope  ports  and  access  panels. 
The  inspection  uses  an  optical  probe  to  provide  a  visual  inspection  of  turbomachinery.  It 
is  meant  to  provide  maintainers  a  means  to  detect  any  damage  that  may  have  occurred  to 
engine  components  in  the  gas  path.  A  CBM+  CONOPS  might  eliminate  the  need  to 
perform  this  inspection  based  on  time  intervals  and  would  rely  on  sensors  and  parametric 
data  trending  to  drive  this  maintenance  activity  based  on  the  actual  condition  of  the 
engine.  This  will  enable  the  maintainers  to  make  informed  decisions  as  to  when  they 
should  perform  borescope  inspections. 

2  FOD  can  be  generated  from  personnel,  airport  infrastructure  (pavements,  lights,  and  signs),  the 
environment  (wildlife,  snow,  ice)  and  the  equipment  operating  on  the  airfield  (aircraft,  airport  operations 
vehicles,  maintenance  equipment,  fueling  trucks,  other  aircraft  servicing  equipment,  and  construction 
equipment)  [15]. 

3  Maintenance  is  essential  to  aviation  safety,  yet  improper  maintenance  contributes  to  a  significant 
proportion  of  aviation  accidents  and  incidents.  This  is  because  a  small  percentage  of  maintenance  tasks  are 
performed  incorrectly  or  are  omitted  due  to  human  error.  Examples  include  parts  installed  incorrectly, 
missing  parts,  and  the  omission  of  necessary  checks  [16]. 
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The  duct  dive  is  another  proactive  maintenance  inspection  used  in  legacy  tactical 
fighters.  During  this  procedure,  the  maintenance  technician  climbs  through  the  aircraft 
inlet  to  inspect  the  inlet  guide  vanes  and  the  first  few  stages  of  the  engine  fan,  as  seen  in 
Figure  6.  The  purpose  of  the  inspection  is  to  visually  and  tactilely  inspect  fan  blades  for 
cracks  that  may  be  propagating  in  the  airfoil.  This  can  also  be  a  time-consuming 
maintenance  check  and  there  is  potential  for  maintenance-induced  damage  (tools  left 
behind,  inlet  surface  damage,  etc.).  Similar  to  the  borescope  inspection,  this  time-based 
maintenance  would  be  eliminated  in  a  CBM+  CONOPS  and  be  driven  by  some 
automated  means  of  impending  failure  detection. 


Figure  8.  Inlet  Inspection  of  E/A  6B  and  F-16.  From  [17] 

Propulsion  systems  on  legacy  platforms  will  generally  service  the  lube  system 
after  each  flight.  This  procedure  involves  topping  off  the  oil  tank  and  checking  magnetic 
chip  collectors.  The  chip  collectors  provide  an  indication  of  machine  wear,  which  can  be 
a  precursor  to  component  failure  (gears,  bearings,  etc.).  Oil  samples  are  also  taken  at 
regular  intervals  and  analysis  of  the  lubricant  is  performed  to  detect  wear  material  and  to 
determine  the  condition  of  the  oil.  In  a  CBM  CONOPS,  these  tasks  would  be  performed 
according  to  need  rather  than  scheduled  intervals.  The  lube  level  would  be  monitored  and 
replenished  based  on  limits  set  by  the  system  designer,  while  the  chip  detection  and  oil 
analysis  would  be  replaced  by  an  online  oil  system  monitor. 
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In  addition  to  proactive  inspections,  critical  propulsion  system  components  will 
also  undergo  proactive  replacement.  Once  life-limited  parts  have  reached  a  particular 
time  on  wing,  they  are  replaced  regardless  of  their  current  state  of  health.  This  system 
management  policy  often  leads  to  discarding  perfectly  good  hardware.  A  condition-based 
maintenance  CONOPS,  which  enables  the  health  assessment  of  components,  based  on 
their  actual  usage  and  life  remaining,  would  drive  down  sustainment  costs  and  increase 
fleet  readiness.  In  order  to  implement  a  CBM  scheme,  a  PHM  system  would  need  to 
sense,  detect,  and  track  the  health  of  the  propulsion  system.  PHM  would  then  drive  these 
proactive  maintenance  actions  based  on  the  actual  condition  of  the  hardware,  as  opposed 
to  the  length  of  operation.  An  automated  means  of  monitoring  hardware  would  need  to  be 
implemented  in  order  to  drive  a  condition-based  maintenance  scheme. 

C.  PHM  TECHNOLOGY  ASSESSMENT 

As  stated  above,  matching  available  hardware,  software,  and  technologies  is  one 
of  the  most  difficult  tasks  for  systems  engineers.  The  CBM+  Guidebook  also  states  that 
no  combination  of  technology  is  likely  to  provide  the  “perfect”  solution  and  the  team  will 
need  to  make  numerous  compromises,  trading  off  required  capabilities  against  cost,  time, 
and  implementation  difficulty  [5].”  The  proof  of  principle  demonstration  discussed  above 
is  a  useful  tool  for  assessing  available  technologies  for  PHM  systems  and  aiding  the  team 
in  arriving  at  the  right  combination  of  technologies. 

The  CBM+  guidebook  states  that  “due  to  time  and  funding  resources  required  for 
CBM+  implementation,  it  is  highly  advisable  for  implementers  to  accomplish  small-scale 
demonstrations  of  primary  CBM+  methods  and  technologies  before  full-scale 
implementation.  A  short-tenn  pilot  test  that  uses  equipment  likely  to  be  used  for  later  full 
implementation  can  be  a  low  risk  approach  to  ensuring  the  feasibility  and  benefits  of  the 
desired  capabilities  [5].” 

To  assess  the  available  and  emerging  PHM  technologies  for  the  F-35  propulsion 
system,  the  JSF  Program  performed  a  sensor  evaluation  during  the  Concept  Demonstrator 
Phase  (CDP).  During  this  test,  candidate  PHM  technologies  were  placed  on  an  F-100 
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engine  during  seeded  fault  testing.3 4  Figure  7  illustrates  some  of  the  candidate 
technologies  used  during  SFET.  These  technologies  were  evaluated  for  their  ability  to 
detect  various  failure  modes  and  seeded  fault  events  during  the  propulsion  system  test. 
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Figure  9.  PHM  Sensors  Evaluated  During  SFET.  From  [6] 


The  results  of  the  test  were  used  to  give  the  systems  engineers  a  baseline 
understanding  of  the  technologies  that  were  available  and  the  Technical  Readiness  Level 
(TRL)  of  each  sensor.  Using  the  data  from  the  SFET,  the  program  was  able  to  establish 
requirements  for  the  PHM  system,  requirements  that  would  eventually  enable  a  CBM+ 
CONOPS.  The  development  and  maturation  of  a  PHM  system,  which  enables  a  CBM+ 

3  Seeded  fault  testing  involves  an  engine  test  where  failure  modes  are  intentionally  excited  for  the 

purpose  of  evaluating  detection  or  accommodation  capabilities  of  the  PHM  system  or  control  system  [6]. 
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CONOPS,  is  potentially  the  longest  lead  procurement  item  for  modern  weapon  systems. 
The  SFET  testing  made  Program  Management  aware  of  the  technologies  that  could  be 
integrated  in  the  near  term,  but  they  still  needed  to  plan  and  budget  for  advances  in  the 
sensing  and  computing  capabilities  that  evolve  during  maturation. 

D.  STAKEHOLDER  DRIVEN  REQUIREMENTS 

Thus  far  this  chapter  has  established  the  legacy  maintenance  procedures  for 
military  propulsion  systems  and  an  approach  to  executing  a  technology  assessment. 
Systems  engineers  developing  a  PHM  system  can  use  this  information  to  proceed  with 
the  development  of  stakeholder  driven  requirements. 

There  are  several  different  types  of  requirements  identified  in  the  SEF,  but  since 
the  scope  of  this  research  is  limited  to  a  notional  PHM  system,  the  functional 
requirements  are  the  most  applicable.  The  SEF  defines  functional  requirements  as  “the 
necessary  task,  action  or  activity  that  must  be  accomplished.  It  also  states  that  functional 
requirements  must  be  achievable  and  must  reflect  a  need  or  objective  for  which  a  solution 
is  technically  achievable  [13].”  To  simplify  this  definition,  we  need  to  establish  a 
necessary  task  with  an  achievable  solution.  Within  the  scope  of  this  research,  functional 
requirements  are  deemed  necessary  tasks  by  examining  the  legacy  maintenance 
procedures  the  PHM  system  is  being  designed  to  replace.  This  research  also  examines  a 
technical  assessment  of  PHM  sensors  used  to  identify  an  achievable  solution. 

Based  on  the  legacy  schedule-based  maintenance  practices  identified  above,  the 
necessary  tasks  of  a  notional  PHM  system  are  listed  across  the  top  of  Table  1.  These 
tasks  are  to  eliminate  scheduled  inspections  and  to  provide  diagnostic  and  prognostic 
assessments  of  the  health  of  the  propulsion  system.  The  achievable  solutions  are  listed 
vertically  and  were  taken  from  the  technology  assessment  outline  above  in  Figure  9. 
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Table  1.  HM  Functions  Mapped  to  Goals 

The  necessary  tasks  and  achievable  solutions  identified  in  Table  1  are  not 
complete,  but  offer  a  good  basis  for  the  definition  of  high-level  PHM  system 
requirements.  The  purpose  of  Table  1  is  to  provide  a  mapping  of  tasks  to  solutions.  This 
connectivity  provides  systems  engineers  with  insight  to  the  design  interfaces  and  enables 
informed  decisions  regarding  requirements.  For  example,  FOD/DOD  Detection  and 
Combustor  Erosion  are  each  mapped  to  the  goal  of  Borescope.  The  identification  of  this 
design  interface  along  with  a  criticality  analysis  from  the  FMECA  can  offer  Systems 
Engineers  increased  coverage  and  confidence  for  the  detection  of  certain  failure  modes. 

E.  PHM  SYSTEM  ARCHITECTURE 

As  stated  in  Chapter  II,  the  requirements  loop  of  the  System  Engineering  Process 
is  followed  by  Functional  Allocation.  This  section  will  attempt  to  refine  high-level  PHM 
system  requirements  to  into  detailed  functional  and  performance  design  criteria.  The 
principles  used  for  this  discussion  on  PHM  architecture  are  based  on  course  material 
provided  by  Professor  John  Osmundson,  Naval  Postgraduate  School  [18]. 
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To  explore  the  development  of  a  PHM  system  architecture,  it  would  again  be 
useful  to  frame  the  discussion  with  Professor  Gary  Langford’s  definition: 

The  system  architecture  model  is  a  cohesive  statement  of  the  system’s 
physical  configuration  in  terms  of  modules,  the  information  flow  between 
them,  and  interconnects  [14]. 

Professor  Osmundson  states  that  “the  first  step  in  establishing  a  system 
architecture  for  PHM  is  to  create  a  high-level  and  solution-neutral  functional 
decomposition.  A  solution-neutral  characterization  of  the  system  will  enable  the  system 
architect  to  view  the  entire  problem  for  what  it  is  and  not  how  to  solve  it.  This  approach 
also  allows  the  architect  to  identify  the  proper  solution,  which  may  not  necessarily  be  the 
first  solution  [18].”  Figure  10  illustrates  a  generic  functional  decomposition  of  a  PHM 
system. 


PHM  System 

Data 

Acquisition  Diagnostics 

Prognostics 

Health 

Management  Testability 

Figure  10.  PHM  Functional  Decomposition 

The  data  acquisition  function  will  be  needed  to  obtain  information  from  the  suite 
of  onboard  sensors  that  provides  data  regarding  the  health  of  the  engine  components.  This 
data  will  be  processed  using  tailored  diagnostic  and  prognostic  algorithms  to  provide  an 
assessment  of  the  current  state  of  the  propulsion  system,  as  well  as  a  prediction  of  life 
remaining  for  all  life-limited  components.  The  Health  Management  module  will  combine 
all  the  various  data  sources  and  use  them  within  an  overarching  logistics  system  to 
provide  work  orders  for  inspection,  repair  and  replacement  of  engine  components. 
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Finally,  the  Testability  function  will  provide  the  capability  for  the  PHM  System  to 
perform  an  autonomous  built-in  test  (BIT),  to  provide  verification  of  the  functionality  of 
the  system. 

The  functional  decomposition  in  Figure  10  offers  no  particular  means  to  achieve  a 
system;  it  is  only  meant  to  frame  the  problem.  The  next  step  is  to  further  decompose  the 
solution  where  it  helps  to  start  by  modeling  the  system  as  a  single  black  box,  operating  on 
material,  energy,  and  signal  flows.  The  black  box  decomposition  is  shown  in  Figure  1 1 . 
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Figure  1 1 .  PHM  Black  Box  Decomposition 

To  further  refine  the  black  box  model,  it  is  decomposed  two  more  layers. 
According  to  Professor  Osmundson,  this  decomposition  will  “offer  insight  to  the 
interfaces  and  functionality”  of  the  system  [18].  The  illustration  in  Figure  12  outlines  the 
refined  functional  decomposition.  The  colored  lines  in  Figures  11  and  12  represent  the 
transfer  of  material  (blue),  energy  (red),  and  data  (dashed  blue).  The  material  input  is 
represented  by  Pilot  Debrief;  after  the  flight  the  operator  feedback  is  recorded  in  the 
PHM  System.  The  energy  input  is  represented  by  the  suite  of  PHM  hardware.  This 
hardware  consists  of  sensors  and  electronics  used  to  detect  faults  in  the  system,  while  the 
data  path  processes  this  energy  to  provide  input  back  to  the  energy  path.  This 
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interdependency  is  represented  by  a  feedback  loop  between  the  energy  and  data  inputs. 
These  three  inputs  flow  to  the  electronics  function  and  result  in  the  system  output: 
System  Health  Status. 


Figure  12.  Refined  Functional  Composition 


F.  SUMMARY 

The  intent  of  CBM+  systems  is  to  replace  legacy  maintenance  practices.  Chapter 
III  identified  different  types  of  maintenance  perfonned  on  legacy  propulsion  systems. 
The  most  common  routine  inspections  and  proactive  maintenance  actions  for  these 
systems  were  borescope  inspections,  duct  dives,  and  oil  sampling. 

The  Seeded  Fault  Engine  Test  used  by  the  F-35  Program  was  presented  as  a 
strategy  to  assess  available  technologies  when  establishing  requirements  for  a  PHM 
system.  The  test  designers  used  PHM  technologies  to  target  schedule-based  inspections. 
They  also  used  FMECA  information  to  target  specific  failure  modes.  This  test  gave  the 
Program  Managers  a  baseline  understanding  of  the  PHM  technologies  that  were  available 
and  the  relative  reliability  and  maturity  of  each  sensing  system. 
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As  discussed  in  Chapter  II,  the  CBM+  requirements  are  based  on  FMECA 
analysis  and  PHM  technologies.  An  understanding  of  the  legacy  maintenance  practices, 
along  with  knowledge  of  available  technologies,  will  enable  acquisition  professionals 
within  the  DoD  to  define  achievable  stakeholder  driven  requirements.  Defining 
achievable  requirements  should  be  the  focus  of  every  system  designer.  Weapon  systems 
that  are  developed  to  meet  performance  levels  beyond  the  current  achievable  capability 
often  suffer  schedule  and  cost  impacts.  This  concept  will  be  further  discussed  in 
subsequent  chapters. 

The  CBM+  Guidebook  provides  methods  for  establishing  requirements  and 
Professor  Osmundson  offers  a  technique  for  developing  solution  neutral  architectures. 
These  methods  were  examined  and  tailored  to  PHM  systems  in  order  provide  system 
developers  with  a  reference  and  tool  for  guidance  during  this  critical  development  stage. 
The  CBM+  Guidebook  also  offers  several  questions  that  CBM+  program  managers  and 
systems  engineers  can  use  as  a  checklist  for  developing  CBM+  systems. 

1.  Do  I  have  sufficient  background  information  on  CBM+  to  assess  the 
current  maintenance  program  in  my  organization  regarding  this  strategy? 

2.  Does  the  CBM+  implementation  team  fully  understand  the  reasons  for 
transition  from  current  maintenance  approaches  to  a  CBM+  environment? 

3.  Is  additional  research  needed  to  familiarize  myself  and  team  members 
with  CBM+  background,  policies,  technologies,  or  other  relevant 
information  [5]? 

These  questions  can  be  used  within  the  scope  of  this  research  to  provide  a 
roadmap  for  developing  the  requirements  for  a  PHM  system.  Each  of  these  questions  are 
addressed  in  this  thesis  with  an  examination  of  legacy  maintenance  practices  for  military 
propulsion  systems  and  in  subsequent  chapters  with  a  discussion  of  strategies  for  gaining 
program  wide  support  for  the  implementation  of  a  disruptive  technology. 

The  Propulsion  PHM  architecture  is  a  highly  integrated  and  complex  system 
designed  to  enable  a  CBM+  CONOPS.  By  its  nature,  PHM  is  a  system  of  systems  that 
requires  integration  across  each  Integrated  Product  Team  (IPT)  within  the  propulsion 
system.  The  development  of  the  PHM  system  not  only  falls  to  the  PHM  system 

developers,  but  also  to  each  component  owner  across  the  entire  weapon  system.  The 
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successful  implementation  of  CBM+  requires  that  the  systems  architect  and  systems 
engineers  work  to  provide  a  comprehensive  overarching  architecture  and  traceability  for 
each  requirement  throughout  every  component  and  IPT.  This  chapter  offered  some 
techniques  for  developing  that  architecture  through  a  solution  neutral  approach  consisting 
of  various  levels  of  requirements  decompositions  and  the  allocation  of  functions  to 
system  goals. 
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IV.  PHM  SOFTWARE  DEVELOPMENT 


A.  INTRODUCTION 

Once  the  system  architecture  has  been  established,  the  software  design  will 
represent  much  of  the  work  remaining  to  implement  a  CBM+  capable  system.  This 
chapter  will  discuss  the  principles  of  software  development  related  to  PHM  and  identify 
some  common  practices  and  lessons  learned. 

B.  SYSTEMS  ENGINEERING  PROCESS  FOR  PHM  SOFTWARE 

DEVELOPMENT 

Thus  far,  the  research  has  been  focused  on  hardware  requirements  and 
architecture  development  for  PHM  systems.  The  software  development  activities  for 
PHM  systems,  however,  can  represent  the  most  potential  for  cost  and  schedule  overruns. 
According  to  a  GAO  report  on  software  acquisition,  “a  review  of  five  DoD  programs 
found  that  outcomes  were  mixed  for  software-intensive  acquisitions.  The  F/A-18  C/D,  a 
lighter  and  attack  aircraft,  and  the  Tactical  Tomahawk  missile  had  fewer  additional  cost 
and  schedule  delays.  For  these  programs,  developers  used  an  evolutionary  approach, 
disciplined  processes,  and  meaningful  metrics.  In  contrast,  the  following  programs, 
which  did  not  follow  these  management  strategies,  experienced  schedule  delays  and  cost 
growth:  F/A-22,  an  air  dominance  aircraft;  Space-Based  Infrared  System,  a  missile- 
detection  satellite  system;  and  Comanche,  a  multi-mission  helicopter  [19].”  This  section 
will  examine  the  evolutionary  approach  and  some  of  the  processes  and  metrics  that 
helped  the  F/A-18  C/D  and  Tomahawk  missile  program  deliver  software  products  with 
fewer  additional  cost  and  schedule  delays. 

The  design  and  development  of  software  products  can  be  characterized  as  an 

iterative  process.  This  concept  is  universally  applicable  software  products  for  PHM 

systems.  These  systems  generally  take  more  time  to  mature,  because  not  all  faults  will 

present  themselves  during  testing.  While  FMECA  driven  diagnostics  systems  are  often 

comprehensive,  there  will  always  be  unknown  -  unknowns.  For  this  reason,  the  software 

systems  engineering  design  process  followed  for  the  development  of  these  modules  is 
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most  often  an  Evolutionary  Systems  Engineering  Model.  This  model  allows  designers  to 
progress  through  the  basic  software  development  life  cycle  multiple  times,  adding 
functionality  and  lessons  learned  with  each  successive  release.  A  DoD  memorandum  on 
software  development  states  that  “evolutionary  acquisition  and  spiral  development  area 
methods  will  allow  us  to  reduce  our  cycle  time  and  speed  the  delivery  of  advanced 
capability  to  our  warfighters.  These  approaches  are  designed  to  develop  and  field 
demonstrated  technologies  for  both  hardware  and  software  in  manageable  pieces  [20].” 

The  illustration  in  Figure  1 1  depicts  an  evolutionary  systems  engineering  model. 
The  process  begins  with  user  requirements  and  progresses  from  design  through  test.  At 
the  end  of  each  cycle,  feedback  or  lessons  learned  from  Operations  are  documented  and 
the  iteration  process  begins.  After  each  iteration,  the  evolutionary  process  is  followed 
while  adding  functionality  and  correcting  issues  identified  in  previous  versions  of  the 
logic. 
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Figure  13.  Evolutionary  Systems  Engineering  Model.  From  [14] 


The  GAO  met  with  industry  software  developers  to  “identify  disciplined 
approaches  to  software  development  [19].”  Figure  12  highlights  the  four-gated  reviews 
often  used  during  the  software  development  activity.  The  processes  outlined  in  this  chart 
exist  within  the  evolutionary  approach  outline  in  the  previous  section.  According  to  the 
GAO,  “within  each  phase  are  key  activities  that  must  take  place  and  knowledge,  or 
information,  that  must  be  attained  to  pass  a  review  and  move  to  the  next  phase  of 
development  [19].” 
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Figure  14.  Knowledge  Based  Software  Development  Process.  From  [19] 

The  system  requirements  are  established  and  approved  at  Gate  1 .  As  discussed  in 
Chapter  II,  the  systems  engineering  process  can  be  decomposed  to  requirements  and 
allocation.  Therefore,  this  is  a  critical  review  that  will  establish  the  scope  of  the  entire 
project.  There  will  likely  be  some  changes  to  requirements  during  the  development  of 
most  software  development  projects,  but  the  Gate  1  review  should  establish  the  expected 
level  of  effort  for  the  entire  program.  The  GAO  states  that  “software  developers  typically 
devote  about  20%  to  30%  of  their  software  development  time  to  requirements-setting 
activities.  Doing  so  ensures  that  developers  will  be  able  to  provide  managers  with  key 
knowledge  at  the  requirements  review  gate  and  show  that  requirements  have  been 
properly  vetted  with  the  acquirer  and  that  they  are  achievable  and  well  written  [19].” 

The  software  design  review  occurs  at  Gate  2.  Figure  14  shows  that  at  this  stage 
the  software  design  documents  and  test  plans  are  completed  and  reviewed.  The  GAO 
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states  that  “as  with  typical  hardware  reviews,  Gate  2  is  often  broken  down  by  preliminary 
and  critical  design  review.  A  stable  design  ensures  that  all  requirements  are  addressed 
and  that  components  and  interfaces  are  defined  [19].” 

Once  the  software  has  passed  the  design  reviews  the  coding  begins  in  preparation 
of  Gate  3.  The  GAO  states  that  “at  this  stage,  the  design  documents  are  used  for  logic 
development  by  software  engineers.  The  success  of  this  stage  often  relies  upon  the  ability 
of  the  software  engineer  to  interpret  the  requirements  established  during  design. 
Ambiguous  design  requirements  will  often  lead  to  re-work  and  result  in  added  cost  and 
schedule  delays  at  this  stage  [19].” 

The  final  stage  of  the  process  is  testing.  In  their  report  the  GAO  states  that 
“testing  is  then  performed  to  uncover  defects  or  gaps  in  the  code.  Leading  software 
companies  we  visited  develop  test  plans  after  requirements  are  stable  and  take  steps  to 
ensure  that  there  are  one  or  more  tests  for  each  requirement  [19].” 

Another  software  development  process  used  by  DoD  is  the  Work  Breakdown 
Structure  (WBS).  The  Military  Standard  881 A  states  that  “the  WBS  identifies  the  product 
to  be  developed  and/or  produced.  It  relates  the  elements  of  work  to  be  accomplished  to 
each  other  and  to  the  end  product.  A  WBS  can  be  expressed  to  any  level  of  detail; 
however,  the  top  three  levels  are  the  minimum  recommended  any  program  or  contract 
needs  for  reporting  purposes  unless  the  items  identified  are  high  cost  or  high  risk  [21].” 
The  illustration  in  Figure  13  represents  a  generic  WBS  for  a  PHM  Software  IPT.  The 
elements  are  identified  for  three  levels  to  gain  perspective  on  the  scope  of  the  project. 
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Diagnostic  Software  WBS  1.1 


Figure  1 5 .  Notional  PHM  WBS 

The  MUIRS  analysis  is  another  software  procurement  process  used  by  the  DoD. 
Professor  Dave  Matthews  states  that  “a  MUIRS  analysis  is  performed  using  the  WBS  to 
identify  critical  support  functions  within  the  software  [22].”  The  MUIRS  Key  in  Figure 
13  identifies  the  WBS  elements  according  to  their  supportability  functions.  The 
Maintainability  function  will  be  the  responsibility  of  the  Software  Requirements  element. 
This  element  will  ensure  that  all  software  maintenance  requirements  are  established  to 
support  the  product  throughout  its  life  cycle.  All  upgrades  to  the  software  will  reside 
within  the  Software  Change  Request  element.  This  element  will  need  to  establish  a 
protocol  for  new  development,  testing,  and  implementation  of  all  new  software 
requirements.  The  Weapon  System  Integration  element  will  manage  all  software 
interfaces.  There  are  only  a  few  interfaces  listed  in  the  example  WBS,  but,  in  reality,  the 
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Propulsion  PHM  software  will  have  many  interfaces  and  any  changes  to  the  product 
could  have  a  cascading  effect  across  the  weapon  system.  The  software  reliability  function 
of  the  MUIRS  analysis  is  identified  within  both  the  Fault  Detection  and  Fault  Isolation 
WBS.  At  this  level,  the  perfonnance  specification  language  is  developed.  For  example, 
the  Fault  Detection  and  Fault  Isolation  requirements  for  reliability  could  be  established  at 
90%  and  75%,  respectively.  Finally,  the  Security  &  Safety  element  of  the  WBS  provides 
these  functions  during  development  and  throughout  the  sustainment  of  the  diagnostic 
software. 

The  Maintainability  portion  of  the  MUIRS  analysis  could  be  considered  to  be  the 
most  critical  for  PHM  software  development.  Software  support  is  by  far  this  biggest  life 
cycle  cost  driver  and  the  most  significant  component  of  system  risk.  The  ability  to 
support  major  software  intensive  systems  is  a  paramount  mission  requirement.  According 
to  Professor  Brad  Naegle  “the  future  capability  of  major  systems  in  a  net-centric  warfare 
environment  is  totally  dependent  on  the  ability  to  cost-effectively  maintain  them  [23].” 
As  stated  in  Chapter  I,  the  DoD  has  targeted  O&S  costs  through  better  systems 
engineering  practices  and  acquisition  training.  The  O&S  costs  for  weapon  systems 
represent  the  65%  to  80%  of  the  life  cycle  costs  of  weapon  systems.  Professor  Naegle 
also  states  that  “within  that  percentage  of  O&S  costs,  the  cost  to  maintain  software  is 
typically  between  60%  and  80%  of  the  software  component  total  life  cycle  cost  (LCC) 
[23].”  Software  maintenance,  therefore,  has  the  potential  to  provide  tremendous  LCC 
savings  to  the  DoD  if  the  acquisition  practices  and  development  can  be  improved. 
Software  maintenance  costs  will  improve  if  we  are  able  to  develop  and  initially  field 
more  mature  software  products.  The  responsibility  for  delivering  mature  systems,  as 
previously  stated,  remains  with  the  Program  Managers  and  Systems  Engineers  within  the 
DoD. 

The  final  characteristic  of  successful  software  acquisition  programs  identified  by 
the  GAO  was  the  proper  use  of  metrics.  The  GAO  interviewed  several  commercial 
software  development  firms  to  understand  the  metrics  commonly  used  by  industry.  They 
identified  the  following  seven  metrics  for  managing  software  development  activities: 
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Cost,  Schedule,  Size,  Requirements,  Tests,  Defects,  and  Quality.  Using  metrics  for  cost 
and  schedule  is  pretty  common  in  the  DoD.  Large  acquisition  programs  now  require 
some  sort  of  cost  and  schedule  control  and  reporting  system.  The  Earned  Value 
Management  System  (EVMS)  is  used  by  many  government  agencies  and  the  private 
sector.  Program  Managers  need  to  be  very  familiar  with  EVMS  and  lean  on  their  Budget 
and  Finance  Managers  for  insight. 

The  size  metric  is  often  referred  to  as  Source  Lines  of  Code  (SLOC).  SLOC  is  the 
number  of  lines  of  text  within  source  code.  SLOC  overruns  have  the  potential  to  drive  a 
great  deal  of  cost  into  a  program.  The  processing  and  throughput  requirements  for 
electronics  (i.e.,  FADEC)  are  partly  based  on  SLOC  estimates.  The  redesign  of  this 
hardware  due  to  requirements  creep  and  SLOC  overruns  is  very  costly.  This  is  an  area 
where  the  evolutionary  concept  differs  for  hardware  and  software,  because  you  cannot 
necessarily  evolve  hardware  incrementally  the  way  you  can  with  software. 

Establishing  requirements  metrics  early  in  the  acquisition  phase  is  an  essential 
component  of  meeting  cost  and  schedule  targets.  This  requirement  metric  is  used  not  only 
to  track  the  addition  of  new  requirements,  but  also  their  timing.  This  is  important  because 
changes  are  easier  and  less  costly  to  implement  in  the  early  stages  of  development.  New 
requirements  that  arrive  late  in  development  will  drive  greater  cost  into  the  program.  The 
metric  can  be  used  to  alert  Program  Management  of  these  potential  cost  drivers. 

The  final  metrics  identified  by  GAO  were  tests,  defects  and  quality.  These  metrics 
provide  management  with  feedback  regarding  the  software  that  is  in  the  latter  stages  of 
development.  Issues  identified  by  these  metrics  could  impede  the  evolutionary 
development  cycle  and  cause  delays  with  subsequent  versions  of  code  and  additional 
capabilities. 

C.  SUMMARY 

This  chapter  also  discussed  the  principles  of  software  development  related  to 
PHM  and  identified  some  common  practices  and  lessons  learned.  The  GAO  identified 
three  industry  best  practices  for  the  development  of  software  systems:  an  evolutionary 
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approach,  disciplined  processes,  and  meaningful  metrics.  The  evolutionary  approach 
employed  basic  systems  engineering  process  to  establish  requirements,  design,  build,  and 
test  the  software.  The  disciplined  process  included  use  of  gated  reviews  at  each  stage  of 
software  development.  This  research  also  identified  and  explored  the  use  of  the  WBS  and 
MUIRS  analysis  as  a  necessary  process  for  software  development  and  sustainability.  The 
metrics  commonly  used  by  industry  for  software  development  were:  Cost,  Schedule, 
Size,  Requirements,  Tests,  Defects,  and  Quality.  The  F/A-18  C/D  and  Tomahawk  Missile 
were  identified  as  two  DoD  acquisition  programs  that  were  able  to  deliver  their  products 
near  cost  and  schedule  targets  by  using  these  industry  best-practices. 
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V.  PHM  ACQUISITION  STRATEGY 


A.  INTRODUCTION 

This  chapter  will  identify  some  strategies  for  acquisition  managers  regarding  the 
procurement  of  PHM  systems.  These  highly  integrated  and  complex  systems  are  a 
relatively  recent  requirement  for  implementation  on  DoD  propulsion  systems.  This 
chapter  will  discuss  a  legacy  program  that  was  able  to  field  and  upgraded  PHM  system 
and  identify  some  of  the  lessons  learned  and  best  practices  related  to  this  successful 
acquisition  program.  Chapter  IV  will  also  examine  some  of  the  obstacles  facing  the 
program  managers  that  procure  PHM  systems  with  the  DoD.  Finally,  the  chapter  will 
wrap  up  with  a  discussion  of  the  maturation  process  for  a  PHM  system. 

B.  F/A-18  E/F  IMPLEMENTATION  OF  PHM 

The  Innovator ’s  Dilemma  written  by  Clayton  Christensen  introduces  the  concept 
of  sustaining  and  disruptive  technologies.  An  MIT  website  offers  a  summary  of  the  book 
and  states  that  “sustaining  technologies  are  technologies  that  improve  product 
performance.  These  are  technologies  familiar  to  most  large  companies;  technologies  that 
involve  improving  a  product  that  has  an  established  role  in  the  market.  Disruptive 
technologies  are  innovations  that  result  in  worse  product  performance,  at  least  in  the  near 
tenn.  Disruptive  technologies  occur  less  frequently;  when  they  do,  however,  they  can 
cause  the  failure  of  highly  successful  companies  that  are  only  prepared  for  sustaining 
technologies  [24].”  Christensen’s  theories  were  developed  based  on  the  application  of 
new  technologies  by  businesses  competing  for  market  share,  but  using  an  abstract 
interpretation  of  these  concepts,  the  principles  of  sustaining  and  disruptive  technologies 
can  be  applied  to  the  implementation  of  CBM+  systems  within  the  military. 

For  the  purpose  of  this  interpretation,  sustaining  technologies  are  analogous  to  the 
legacy  CONOPS  for  aviation  maintenance,  i.e.,  schedule-based  inspections,  schedule- 
based  removals,  etc.  The  technologies  that  enable  these  maintenance  practices  have 
consistently  been  developed  and  improved  over  time.  The  Navy  F/A-18  E/F  Super 
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Hornet  acquisition  program  is  a  good  example  of  a  sustaining  technology  improvement 
for  Propulsion  PHM  and  can  serve  to  put  the  application  of  this  theory  in  context. 


Figure  16.  Super  Hornet  approach  aboard  CVN  72.  Photo  taken  by  Author. 

The  US  Navy  and  Boeing  collaborated  on  a  paper  about  the  upgrade  of  the  F/A- 
18  propulsion  monitoring  system.  The  paper  states  that  “under  Navy  direction,  GE  and 
Boeing  have  enhanced  the  proven  performance  of  the  F/A-18  C/D  F404  In-flight  Engine 
Condition  Monitoring  System  (IECMS)  and  produced  an  Advanced  IECMS  for  the  F/A- 
18  E/F  tailored  for  the  F414  engine.  The  advanced  IECMS  system  is  fully  integrated 
between  the  engine  and  airframe  and  effectively  uses  available  avionics  computers  and 
interfaces,  which  contributes  to  low  system  weight.  This  advanced  system  includes  many 
improvements,  including: 

•  Better  aircrew  displays  and  additional  cautions/advisories, 

•  Additional  mission  computer  resources, 
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•  Reliable,  new  FADEC5  with  outstanding  fault  detection  and  isolation 
capabilities, 

•  Improved  monitoring  hardware  installation  and  signal  processing, 

•  Expanded  memory  unit  data  recording, 

•  Addition  of  an  engine-mounted  master  electrical  chip  detector,  and 

•  Additional  maintenance  codes 

Each  of  these  capabilities  contributes  to  reduced  pilot  workload  and  reduced 
aircraft/engine  maintenance.  The  bottom  line  is  that  aircraft  readiness  is  improved  with 
fewer  engine  runs,  less  down  time  required  for  troubleshooting,  and  rapid  turnaround 
through  onboard  diagnostics  [25].” 

There  were  substantial  improvements  made  to  the  IECMS  during  the  development 
of  the  Super  Hornet  propulsion  system.  The  improvements  were  largely  based  on 
advances  in  computing  resources  and  sensor  capabilities  that  were  developed  as 
sustaining  technologies.  These  technologies  gave  the  maintainers  better  fault  detection 
and  isolation  capability,  but  did  not  substantially  change  the  procedures  for  maintaining 
the  propulsion  system.  In  other  words,  the  improvements  were  not  substantial  enough  to 
bridge  the  gap  between  the  legacy  maintenance  CONOPS  and  CBM+,  but  were  an 
important  step  in  that  direction. 

While  the  F/A-18  E/F  IECMS  upgrade  was  an  important  step  toward  the 
implementation  of  CBM+,  it  did  not  represent  a  paradigm  shift  in  the  way  the  Navy 
maintained  the  propulsion  system.  The  implementation  of  a  CBM+  system  could  be 
characterized  as  a  disruptive  technology.  For  CBM+  to  become  a  reality,  the  Navy  will 
still  need  to  rely  on  advances  in  sustaining  technologies,  but  the  implementation  and 
reliance  upon  these  technologies  will  be  disruptive.  The  greater  reliance  on  new  sensing 
technologies  could  result  in  degraded  performance  in  the  near  term,  as  the  systems  are 
matured.  DoD  maintainers  are  accustomed  to  performing  schedule-based  inspections  and 
removals.  And  while  the  next  generation  of  maintainers  that  will  learn  to  perform 
condition-based  maintenance  and  value  the  disruptive  technology,  the  more  experienced 

5  FADEC — Full  Authority  Digital  Electronic  Control. 
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maintainers  will  likely  resist  the  change,  particularly  in  the  period  during  which  the 
system  is  being  matured  and  product  performance  decreases,  i.e.,  false  alarms,  missed 
detects,  etc. 

The  MIT  review  of  Christensen’s  work  states  that  “disruptive  technologies  cause 
problems  because  they  do  not  initially  satisfy  the  demands  of  even  the  high  end  of  the 
market.  Because  of  that,  large  companies  choose  to  overlook  disruptive  technologies  until 
they  become  more  attractive  pro  fit- wise  [24].”  This  is  illustrated  in  Figure  17. 


Time 


Figure  17.  Disruptive  Vs.  Sustaining  Technologies.  From  [24] 

The  Taguchi  definition  of  quality  and  his  theories  regarding  losses  to  society  are 
relevant  to  this  discussion  of  CBM+  as  a  disruptive  technology.  Taguchi  defined  quality 
as  “the  loss  a  product  causes  to  society  after  being  shipped,  other  than  losses  caused  by 
its  intrinsic  function  [26].”  This  theory  is  analogous  and  applicable  to  the  development  of 
CBM+  systems  within  the  military.  These  systems  are  fundamentally  designed  to  enable 
maintainers,  but  when  not  properly  implemented  or  designed,  they  can  be  burdensome 
and  unreliable.  As  previously  stated,  the  use  of  PHM  systems  to  drive  all  maintenance  is 
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a  huge  paradigm  shift  within  the  DoD  and  change  is  not  always  welcome.  There  is  a 
natural  reluctance  and  skepticism  within  the  maintenance  community  to  rely  on 
automatic  fault  detection  systems  to  drive  maintenance  actions.  Unsuccessful  attempts  in 
the  past  to  implement  these  systems  have  led  to  a  perception  among  some  maintainers 
(society)  that  condition-based  maintenance  may  not  be  an  attainable  goal.  It  is  the 
responsibility  of  Program  Managers  and  Systems  Engineers  to  field  capable  CBM+ 
systems  early  in  the  weapon  system  life  cycle,  such  that  the  system  and  the  overarching 
CONOPS  can  be  optimized  and  matured. 

C.  PHM  MATURATION  PROCESS 

PHM  design,  development,  and  maturation  is  a  unique  process  and  different  than 
all  other  components  on  the  propulsion  system.  The  PHM  System  and  the  associated 
logistics  infrastructure  that  it  supports  are,  potentially,  the  longest  lead  development 
items  associated  with  the  acquisition  of  military  propulsion  systems.  The  diagnostic 
sensors  and  other  hardware  associated  with  the  PHM  system  is  designed,  developed, 
tested,  and  matured  along  with  all  of  the  propulsion  hardware.  However,  this  hardware 
only  provides  a  source  of  data.  The  software  and  algorithms  that  enable  a  CBM+  system 
are  developed  through  an  iterative  process  long  after  the  propulsion  system  has  gone  into 
production. 

As  discussed  previously,  the  technologies  that  enable  a  CBM+  CONOPS  are 
relatively  immature.  The  propulsion  system  FMECA  provides  designers  with  insight  as  to 
the  failure  modes  the  system  should  target,  but  the  verification  and  validation  of  the 
ability  to  detect  these  actual  failures  could,  in  some  cases,  occur  long  after  the  system  is 
fielded.  Ideally,  PHM  system  designers  would  like  to  perform  seeded  fault  testing  to 
validate  their  capability  to  detect  faults  throughout  the  propulsion  system,  i.e.,  the  FI 00 
SFET  testing  discussed  in  Chapter  II.  However,  this  testing  is  cost  prohibitive  in  that  it 
often  leads  to  damaging  expensive  hardware  for  the  sake  of  maturing  a  non-flight  critical 
system.  Therefore,  this  is  an  implausible  approach  for  most  acquisition  programs.  The 
nonnal  timeline  of  verification  and  validation  for  fault  detection  and  fault  isolation 
software  extends  beyond  the  development  phase  and  well  into  production.  Although  this 
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involves  delivering  an  immature  product  to  the  field,  many  of  the  failures  that  the  PHM 
system  is  designed  to  detect  will  present  themselves  during  the  course  of  normal 
operation.  It  is  also  critical  to  update  and  maintain  a  current  and  valid  FMECA  as 
hardware  changes  occur.  The  validity  of  this  data  is  essential  for  RCM  and  CBM+ 
systems. 

The  Taguchi  loss  analogy  cited  in  Chapter  II  stated  that  delivering  an  immature 
PHM  product  to  the  Services  could  have  an  ill  effect  on  the  perception  and  customer 
satisfaction  with  PHM.  However,  due  to  the  unique  requirements  and  long  lead 
maturation  of  the  PHM  system,  the  end  users  actually  become  part  of  the  design  team. 
Once  initially  deployed,  these  maintainers  will  be  able  to  provide  the  system  designers 
with  feedback  regarding  the  operation  of  the  system  and  areas  for  improvement.  It  is, 
however,  critical  that  the  Program  Manager  and  PHM  designers  deliver  a  baseline 
product  that  is  representative  of  the  final  product  with  intennediate  functionality.  If  the 
system  is  deemed  unreliable  by  the  end  user,  the  end-user  will  revert  to  legacy 
maintenance  procedures  and  the  feedback  loop  necessary  to  correct  inherent  system 
design  flaws  will  be  gone. 

D.  PROGRAM  MANAGEMENT  DILEMMA 

Professor  Dave  Matthews  stated  that  “the  number  one  responsibility  of  a  Program 
Manager  is  to  make  a  program  viable  from  a  cost  and  schedule  perspective  [22].”  History 
has  shown  that  weapon  systems  acquisition  budgets  and  schedules  often  suffer  due  to  the 
unknown  unknowns.  The  DoD  procures  some  of  the  most  technically  complex  systems 
designed  by  man,  and  there  are  inevitable  flaws  that  will  present  themselves  throughout 
the  design,  build,  and  test  phase  of  the  acquisition  programs.  This  concept  can  be  applied 
broadly  to  weapon  systems  or  specifically  to  the  development  and  maturation  of 
Propulsion  CBM+  systems. 

The  program  management  dilemma  with  regard  to  the  procurement  of  PHM 
systems  pertains  to  incentive.  With  all  of  the  pressure  placed  on  Program  Managers  to 
make  a  program  viable,  there  is  often  no  sense  of  urgency  or  incentive  to  focus  on 
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systems  that  are  in  place  to  provide  LCC  savings  many  years  into  the  program.  Although 
this  research  has  cited  numerous  reasons  for  providing  PHM  capabilities  as  early  in  the 
program  as  possible,  it  has  become  the  norm  for  Programs  to  mortgage  non-flight  critical 
systems  to  make  their  Program  viable.  The  justification  for  doing  this  is  perfectly 
reasonable.  There  are  political  pressures  that  surround  defense  acquisition  programs  and 
the  technical  challenges  that  the  programs  often  face  drive  program  realignment  activities 
that  focus  on  the  near-term  must  haves.  However,  deferring  PHM  capabilities  comes  at 
the  cost  of  both  LCC  and  increased  development  costs,  due  to  the  fact  that  added 
capabilities  are  always  more  expensive  to  implement  later  in  a  program  life  cycle.  The 
DoD  is  currently  focused  on  these  sustainment  cost  savings,  but  these  savings  will  never 
be  realized  for  PHM  systems  without  programmatic  incentives  for  acquisition  managers 
to  focus  on  LCC.  Without  some  kind  of  incentive,  the  costs  that  any  Program  Manager 
will  focus  on  are  the  here  and  now  costs  to  solve  today’s  technical  issues.  The  LCC 
savings  realized  by  the  implementation  of  a  CBM+  system  will  likely  not  be  realized 
until  well  after  the  Program  Manager  has  moved  on  or  retired.  These  LCC  savings  should 
be  tracked  and  pursued  from  day  one;  currently,  they  are  not.  Program  Managers  are 
forced  to  be  near-sighted  with  regard  to  cost  and  schedule;  this  comes  at  the  detriment  of 
CBM+  systems. 

At  the  program  management  level,  supportability  and  logistics  issues  are  often 
deferred  until  later  in  the  acquisition  program  due  to  limited  resources  and  the  redesign  of 
more  critical  hardware  and  software  systems.  The  prioritization  of  CBM+  through 
programmatic  incentives  will  not  be  a  trivial  policy  to  implement  for  the  DoD,  but  it 
needs  to  be  a  priority  if  the  LCC  savings  associated  with  these  systems  are  to  become  a 
reality. 

E.  SUMMARY 

PHM  systems  are  often  introduced  with  a  performance  degradation  compared  to 
the  legacy  maintenance  practices  they  are  meant  to  improve.  Disruptive  technologies 
have  the  potential  to  transform  the  way  an  organization  operates  and  they  offer  a  great 

deal  of  upside  potential  with  respect  to  O&S  costs.  The  acquisition  professionals  must 
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learn  to  manage  not  only  the  effective  procurement  of  these  systems,  but  also  the 
expectations  of  the  end  user.  These  expectations  were  also  explored  in  a  discussion  of  the 
Taguchi  Loss  Theory.  The  acquisition  managers  must  also  prepare  a  mitigation  strategy 
for  the  potential  perceived  quality  losses  during  the  introduction  of  a  disruptive 
technology. 

The  maturation  and  software  maintenance  of  the  PHM  system  is  likely  to  last  as 
long  as  the  weapon  system  on  which  it  resides.  The  end  user  becomes  an  integral  part  of 
the  design  loop  throughout  this  lifecycle  of  this  product.  Once  again,  the  Taguchi  losses 
experienced  with  this  disruptive  technology  must  be  managed  by  developers  and  Program 
Managers. 

If  the  Life  Cycle  Cost  savings  associated  with  PHM  system  are  to  be  realized,  the 
DoD  must  incentivize  Program  Managers  to  actively  attack  the  enabling  technologies. 
These  technologies  are  often  an  afterthought  due  to  the  need  to  address  more  immediate 
technical  concerns. 
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VI.  CONCLUSION 


A.  INTRODUCTION 

The  purpose  of  this  research  was  to  provide  Systems  Engineers  and  Program 
Managers  a  resource  for  the  acquisition  of  propulsion  PHM  systems.  The  thesis  outlines 
processes  for  establishing  requirements,  defining  a  notional  architecture  and  developing 
software  for  PHM  systems  and  presents  them  within  the  systems  engineering  process.  It 
also  provides  acquisition  strategies  and  lessons  learned  based  on  legacy  PHM  systems. 
The  goal  of  the  research  is  to  enable  the  system  developers  to  use  this  reference  to 
improve  cost  and  schedule  delivery  of  capable  PHM  systems  throughout  the  acquisition 
life  cycle. 

B.  RESEARCH  SUMMARY 

The  research  began  with  a  background  examination  of  military  propulsion  PHM 
systems  and  the  their  evolution  from  the  diagnostic  systems  first  fielded  on  the  A-7 
Corsair  to  the  CBM+  systems  being  developed  for  the  F-35  JSF.  It  also  identified  the 
DoD  directive  that  mandates  the  implementation  of  CBM+  strategies.  This  mandate  was 
an  important  step  in  the  evolution  of  CBM+,  but  in  order  to  be  successfully  implemented, 
there  must  be  a  sense  of  ownership  throughout  each  level  of  the  program.  PHM  is  unique 
in  that  it  touches  each  and  every  IPT  within  the  weapon  system.  Its  development, 
therefore,  is  a  program  wide  responsibility. 

There  are  many  different  systems  engineering  process  models,  but,  generally 
speaking,  they  each  start  with  the  establishment  of  system  requirements.  In  the  DoD, 
requirements  for  weapon  systems  are  commonly  defined  by  the  user  community.  For 
PHM  systems,  it  is  important  to  focus  on  requirements  related  to  the  schedule-based 
maintenance  actions  we  are  seeking  to  transition  to  a  condition-based  CONOPS.  The 
logical  first  step  was  to  examine  current  schedule-based  maintenance  actions.  This 
activity  provides  a  baseline  understanding  for  the  requirements  analysis  needed  to  be 
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performed  by  any  program  planning  to  implement  CBM+.  The  use  of  RCM  tools  and  the 
FMECA  also  provide  an  important  resource  for  identifying  capability  gaps  and 
addressing  them  with  available  PHM  technologies. 

The  definition  of  requirements  was  followed  by  a  high-level  approach  to 
establishing  an  architecture  for  a  generic  PHM  system.  This  method  provides  a  solution 
neutral  characterization  of  the  problem  rather  than  a  way  to  solve  the  problem.  Once  the 
high-level  architecture  is  identified,  it  is  decomposed  to  the  basic  elements,  which  add 
resolution  by  establishing  flows  material,  energy,  and  data. 

There  is  no  one-size-fits-all  approach  to  software  development  and  acquisition. 
This  is  particularly  true  for  the  PHM  software  procurement.  It  is  important  to  focus  on  the 
established  processes  and  metrics  that  have  been  established  and  proven  by  previous 
successful  acquisition  programs.  Research  performed  by  the  GAO  established  some 
lessons  learned  for  software  development.  The  report  suggests  some  methods  the  DoD 
could  employ  to  leverage  industry  best-practices  for  delivering  quality  software  products 
within  time  and  cost  constraints.  The  use  of  an  evolutionary  approach,  disciplined 
processes,  and  meaningful  metrics  were  identified  as  key  enablers  to  successful  software 
acquisition  programs,  such  as  the  F/A-18  C/D  and  Tomahawk  Missile  Program.  Figure 
18  highlights  these  management  practices  and  how  they  relate  to  program  goals. 
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Successful  outcomes: 

Meeting  cost, 
schedule, 
performance, 
and  quality  goals 


Source:  GAO's  analysis  of  leading  companies’  software  development  practices. 


Figure  18.  Key  Management  Practices.  From  [19] 


PHM  was  presented  as  a  disruptive  technology  that  has  the  ability  to  revolutionize 
the  way  the  military  maintains  weapon  systems  and  provide  significant  LCC  savings. 
However,  as  with  any  disruptive  technology,  there  will  be  a  period  of  learning  and  a  need 
to  overcome  the  perceived  or  actual  quality  loss  by  society,  as  outlined  by  Taugchi.  The 
period  of  transition  from  design  and  development  to  initial  demonstration  is  critical  for 
PHM  technologies  to  be  matured.  It  is  important  for  Program  Managers  within  the  DoD 
to  defend  PHM  systems  and  to  manage  expectations  within  the  user  community. 

Most  large  companies  are  adept  at  turning  sustaining  technology  challenges  into 
achievements,  the  F/A-18  E/F  IECMS  upgrade  is  a  good  example  of  this.  IECMS  was  a 
sustaining  tech  upgrade  that  improved  the  F404  engine,  which  had  an  established  role  in 
the  market.  Each  of  the  acquisition  programs  discussed  in  this  chapter  used  a  systems 
engineering  approach  to  the  establishment  of  requirements  for  a  propulsion  PHM  system. 
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The  difference  in  the  way  that  the  F/A-18  and  F-35  programs  approached  their  respective 
evaluation  of  candidate  technologies  is  obviously  much  different.  The  F/A-18  E/F  was  an 
established  platfonn  that  was  upgrading  their  current  diagnostic  capabilities  based  on 
needs  established  by  the  user,  whereas  the  F-35  program  used  the  SFET  program  to  build 
a  CBM+  system  from  the  ground  up.  The  JSF  Program  had  a  need  to  establish  an 
understanding  of  the  technologies  available  at  that  time. 

In  each  case,  the  respective  Programs  started  with  a  customer  need  and  followed 
the  systems  engineering  process  to  deliver  a  product.  Figure  19  identifies  a  basic  systems 
engineering  process  V-  model  commonly  used  for  the  development  of  weapon  systems. 
The  discussion  of  the  Super  Hornet  and  F-35  Programs  focused  on  the  first  two  stages  of 
the  model,  the  Problem  Decomposition  phase  and  the  System  Design  phase,  respectively. 


Customer  Product 


Figure  19.  Systems  Engineering  Process  V-Model.  From  [14] 

The  F/A-18  discussion  focused  on  the  Problem  Decomposition  step  in  the  V- 
model.  The  team  identified  maintenance  issues  in  the  fleet  that  could  be  eliminated  or 
improved  with  the  IECMS  upgrade.  They  interviewed  the  users  to  identify  improvements 
and  establish  requirements  to  increase  maintainability  and  reliability  of  the  F414.  The  F- 
35  example  was  focused  on  the  Systems  Design  phase  of  the  V-model.  This  phase 
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occurred  after  the  Problem  Decomposition  had  been  performed  and  systems  engineers 
had  identified  the  maintenance  actions  they  wished  to  target.  To  begin  the  systems  design 
phase,  they  performed  the  SFET  to  provide  an  analysis  of  alternatives  and  an  assessment 
of  the  TRL  of  existing  PHM  technologies. 

Once  the  PHM  system  is  fielded,  the  maturation  process  begins.  The  support  of 
the  PHM  software  will  last  as  long  as  the  weapon  system  on  which  it  resides.  The 
maturation  of  this  system  is  unique  and  will  often  involve  the  end  user  more  than  other 
systems.  Care  must  be  taken  to  manage  the  Taguchi  losses  experienced  by  maintenance 
personnel  with  regard  to  immature  software  products. 

The  DoD  is  aggressively  looking  for  ways  to  reduce  its  budget  and  weapon 
system  life  cycle  costs  are  a  prime  target.  CBM+  systems  are  designed  to  provide  LCC 
savings,  but  the  technology  is  still  relatively  immature  and  often  not  the  focus  of  Program 
Managers.  The  DoD  needs  to  incentivize  its  Program  Managers  to  focus  on  the 
implementation  of  CBM+  systems  earlier  in  the  weapon  system  development  life  cycle  in 
order  to  capitalize  on  the  LCC  savings  they  provide. 

C.  CONCLUSION  AND  LESSONS  LEARNED 

The  development  of  PHM  will  not  succeed  without  commitment  from  all  levels  of 
the  Program.  The  Program  Management  must  be  willing  to  defend  the  system  during  the 
inevitable  cost-cutting  exercises.  The  system  developers  must  have  a  sense  of 
responsibility  for  the  design  and  integration  activities  that  cross  all  IPT’s.  All  of  the  IPT’s 
have  a  stake  in  the  development  of  the  PHM  system.  The  hardware  and  software 
configuration  management  and  integration  activities  are  extremely  complex  and  errors 
will  have  a  cascading  effect  across  the  weapon  system. 

Program  Management  and  system  developers  are  responsible  for  not  only 
designing  systems  according  to  the  warfighter’s  requirements,  but  also  managing  their 
expectations,  particularly  when  delivering  a  disruptive  technology  with  intennediate 
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functionality.  Programs  must  also  account  for  the  budget  necessary  to  maintain  PHM 
systems  throughout  their  lifecycle.  The  Program  could  consider  the  establishment  of  a 
Software  Maintenance  IPT  at  some  point  during  the  acquisition  phase. 

The  program  manager  and  logic  designers  should  be  aware  of  and  identify  risks 
related  to  the  development  of  PHM  software.  The  following  are  some  common  risks 
associated  with  the  PHM  software  development  activity: 

•  There  is  an  inherent  risk  in  developing  software  for  safety  critical  control 
components.  The  risk  is  especially  high  when  working  with  airborne 
weapon  systems.  The  fault  detection  and  accommodation  logic  must  be 
tested,  verified  and  validated  to  a  high  standard. 

•  Cost  and  schedule  are  always  significant  risk  factors  during  software 
development.  The  WBS  for  these  programs  is  such  that  one  IPT  depends 
on  a  product  from  another  IPT.  There  is  a  risk  of  creeping  requirements. 
The  evolutionary  model  is  used  to  mitigate  this  risk,  but  a  large  change  in 
project  scope  cannot  be  absorbed  or  accommodated  by  any  systems 
engineering  model. 

•  Staffing  issues  and  the  learning  curve  associated  with  developing 
complicated  software  systems  can  place  a  large  amount  of  risk  on 
programs. 

•  The  test  benches  used  for  Verification  and  Validation  (V&V)  testing  will 
often  run  at  100%  capacity  during  peak  times.  A  shortage  of  equipment  to 
test  the  logic  will  create  a  backlog  and  eventually  schedule  delays. 

•  Configuration  control  boards  will  have  their  own  set  of  priorities.  For 
example  changes  to  the  Control  logic  will  get  higher  priority  than  changes 
to  the  Diagnostic  logic.  Excessive  development  issues  with  higher  priority 
modules  will  often  delay  the  development  of  less  critical  modules. 

Tight  collaboration  between  the  customer  and  the  developer  is  a  critical  aspect  of 
controlling  requirements,  cost,  schedule,  and  quality  during  software  projects.  A 
customer  presence  at  the  development  site  can  provide  autonomy  and  a  fast  turn-around 
when  challenges  inevitably  present  themselves.  For  example,  derived  requirements  will 
often  have  some  ambiguity  and  leave  room  for  interpretation.  This  interpretation  could 
impact  project  cost  and  schedule.  However,  having  a  government  representative  with 
decision  making  authority  integrated  with  the  development  team  can  provide  timely 
guidance  and  potentially  eliminate  costly  rework  and  delays. 
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As  stated  above,  the  DoD  is  in  the  business  of  making  the  difficult  seem  easy. 
Schedule  delays  and  cost  overruns  associated  with  developing  high-tech  weapon  systems 
are  nearly  inevitable.  These  vast  and  complex  systems  are  bound  to  experience  shortfalls 
and  unexpected  failures  that  drive  redesign  activities.  However,  it  is  not  unreasonable  to 
expect  better  perfonnance  from  the  DoD  and  the  contractors  that  develop  the  weapon 
systems. 

This  research  provided  examples  of  acquisition  and  development  programs  that 
have  used  systems  engineering  processes  and  industry  best  practices  to  achieve  better 
results.  Along  with  these  documented  improvements  used  on  legacy  programs,  the 
research  also  identified  several  other  success  oriented  criteria  for  delivering  capable  PHM 
systems  closer  to  budget  and  cost  targets,  such  as: 

•  Incentivize  Program  Management  to  focus  on  Life  Cycle  Cost  Savings 

•  Involve  the  end-user  beyond  the  requirements  development  phase 

•  Manage  warfighter  expectations  to  minimize  Taguchi  losses 

•  Plan  to  support  PHM  software  throughout  the  weapon  system  life  cycle 
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